The disclosed embodiments relate generally to panoramic photography. More specifically, the disclosed embodiments relate to techniques for improving panoramic photography for handheld personal electronic devices with image sensors.
Panoramic photography may be defined generally as a photographic technique for capturing images with elongated fields of view. An image showing a field of view approximating, or greater than, that of the human eye, e.g., about 160° wide by 75° high, may be termed “panoramic,” Thus, panoramic images generally have an aspect ratio of 2:1 or larger, meaning that the image being at least twice as wide as it is high (or, conversely, twice as high as it is wide, in the case of vertical panoramic images). In some embodiments, panoramic images may even cover fields of view of up to 360 degrees, i.e., a “full rotation” panoramic image.
Many of the challenges associated with taking visually appealing panoramic images are well documented and well-known in the art. These challenges include photographic problems such as: difficulty in determining appropriate exposure settings caused by differences in lighting conditions across the panoramic scene; blurring across the seams of images caused by motion of objects within the panoramic scene; and parallax problems, i.e., problems caused by the apparent displacement or difference in the apparent position of an object in the panoramic scene in consecutive captured images due to rotation of the camera about an axis other than its center of perspective (COP). The COP may be thought of as the point where the lines of sight viewed by the camera converge. The COP is also sometimes referred to as the “entrance pupil.” Depending on the camera's lens design, the entrance pupil location on the optical axis of the camera may be behind, within, or even in front of the lens system. It usually requires some amount of pre-capture experimentation, as well as the use of a rotatable tripod arrangement with a camera sliding assembly to ensure that a camera is rotated about its COP during the capture of a panoramic scene. Obviously, this type of preparation and calculation is not desirable in the world of handheld, personal electronic devices and ad-hoc panoramic image capturing.
Other well-known challenges associated with taking visually appealing panoramic images include post-processing problems such as: properly aligning the various images used to construct the overall panoramic image; blending between the overlapping regions of various images used to construct the overall panoramic image; choosing an image projection correction (e.g., rectangular, cylindrical, Mercator) that does not distort photographically important parts of the panoramic photograph; and correcting for perspective changes between subsequently captured images.
Accordingly, there is a need for techniques to improve the capture and processing of panoramic photographs on handheld, personal electronic devices such as mobile phones, personal data assistants (PDAs), portable music players, digital cameras, as well as laptop and tablet computer systems. By accessing information returned from positional sensors embedded in or otherwise in communication with the handheld personal electronic device, for example, micro-electro-mechanical system (MEMS) accelerometers and gyrometers, more effective panoramic photography techniques, such as those described herein, may be employed to achieve visually appealing panoramic photography results in a way that is seamless and intuitive to the user.